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Signal-to-Noise  Ratio  and  Central  Limit  Theorem 
Considerations  in  Non-Gaussian  Detection 


Douglas  Warren  and  John  B.  Thomas 
Department  of  Electrical  Engineering 
Princeton  l^niversity 
Princeton,  New  Jersey  08514 


ABSTRACT 

A  generalized  signal-to-noise  ratio  for  detectors  is  defined  and  its  relationship  to  quality  of 
detection  is  investigated.  Specific  attention  is  given  to  log-likelihood  ratio  detectors.  The 
question  is  discussed  as  to  whether  a  Central  Limit  Theorem  approximation  for  the  output 
of  the  detector  gives  results  which  are  close  to  those  of  the  actual  detector  and  is  illustrated 
with  examples. 

I.  INTRODUCTION 


In  this  paper  we  will  be  looking  at  the  discrete  time  problem  of  detection  of  a  con¬ 
stant  signal  in  the  presence  of  independent,  identically  distributed  noise;  that  is. 

Under  H :  A’,-  =  Af,-  i  —  1,2,. ..,n 

and 

Under  K :  Xf  =  Nj  +  s  i  =  l,2,...,n 

where  A',-  is  the  observation,  N,  is  the  noise  and  «  is  a  positive  constant.  The  only  limita¬ 
tions  that  w'e  will  place  on  the  possible  noise  distributions  will  be  that  they  are  symmetric, 
bounded,  unimodal  and  have  zero  means. 

If  the  detector  is  linear,  we  can  define  a  signal-to-noise  ratio  (SNR)  in  terms  of  « ,  n  , 
and  the  noise  variance  cr®.  In  this  case  the  SNR  is  clearly  a  useful  quantity:  for  a  constant 
false  alarm  probability  the  probability  of  detection  is  a  monotonic  function  of  the  SNR.  If 
the  detector  is  nonlinear,  not  only  is  the  relationship  between  SNR  and  detector  perfor¬ 
mance  less  clear  but  it  is  no  longer  immediately  apparent  what  definition  of  SNR  would 
be  useful. 

A  generalized  SNR  has  been  proposed  and  it  has  been  shown  to  be  maximized  when 
the  nonlinearity  is  a  monotonic  function  of  the  likelihood  ratio[l,2].  This  is  an  encourag¬ 
ing  result  since  any  monotonic  function  of  the  likelihood  ratio  is  also  the  Ney man- Pearson 
optimal  detector.  Following  some  unpublished  work  of  Poor,  Chiang  and  Wise[3],  we 
show  later  that,  if  the  sample  size  is  very  large,  this  SNR  functions  much  as  does  the  con¬ 
ventional  SNR  for  the  linear  detector.  In  fact,  it  can  be  shown  that,  asymptotically,  the 
relationship  between  SNR  and  the  probability  of  detection  is  once  again  monotonic. 

As  with  any  asymptotic  result  there  arises  here  the  basic  problem  of  determining 
whether  the  Central  Limit  Theorem  holds  and  the  more  obscure  question  of  deciding  how 
many  samples  are  needed  before  the  implications  of  the  theorem  merge  with  reality.  Obvi¬ 
ously,  the  theorem  is  useless  as  a  practical  tool  if  it  is  necessary  to  collect  an  infinite 
number  of  samples  before  it  can  be  invoked  and  as  a  result  it  is  desirable  to  know  just 
how  much  is  enough.  In  what  follows  we  show  that,  under  some  mild  restrictions,  the  out¬ 
put  of  a  log-likelihood  ratio  detector  obeys  the  Central  Limit  Theorem  and  we  look  at 
some  examples  in  order  to  illustrate  the  rate  of  convergence. 

The  paper  is  composed  of  three  parts  and  an  appendix  containing  proofs.  In  Part  I 
we  define  and  discuss  some  relevant  properties  of  the  generalized  SNR  and  the  log- 
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likelihood  ratio.  In  Part  il  we  investigate  the  relationship  between  SNR  and  traditional 
measures  ci  detector  performance.  In  Part  III  we  look  at  some  numerical  examples  con¬ 
cerning  the  convergence  of  the  log- likelihood  ratio  to  a  Gaussian  random  variable.  Part 
IV  contains  conclusions  and  a  discussion  of  topics  for  further  study. 


I.  PRELIMINARY  PROPERTIES 


The  log-likelihood  ratio.  It  is  well  known[4]  that  the  likelihood  ratio 

maximizes  the  probability  of  detection  0  for  a  given  false  alarm  rate  a.  Here  Ik  and  f  ^ 
are  the  densities  of  the  observation  vector  X  under  the  hypotheses  K  and  H  respectively. 
If  we  are  attempting  to  detect  a  shift  to  the  right  by  a  constant  vector  a  and  /  is  the 
density  of  the  noise  we  have 

/(X-s) 

It  is  also  well  known  that  any  monotonically  increasing  function  of  the  likelihood  ratio  will 
have  the  same  receiver  operating  characteristic  (ROC,  that  is,  0  vs.  a)  as  that  given  by 
the  likelihood  ratio.  The  logarithm  is,  of  course,  one  such  monotonically  increasing  func¬ 
tion.  For  the  case  of  i.i.d.  noise  we  have 

^  /  (  Y.  -s  ) 

/(A-,)  • 

By  using  log  L  rather  than  L  we  not  only  simplify  our  computation  by  converting  the 
product  to  a  sum 

A  /  {  a;  -«  ) 

t  (X)  =--  logL  (X)  =  s  log^ ;  ‘  , 

•  -1  /  (A,) 

but  for  large  enough  n  we  can  approximate  the  test  statistic  /(X)  by  using  the  Central 
Limit  Theorem  and  thereby  further  simplify  the  problem. 

In  this  paper  we  will  discuss  the  log-likelihood  ratio  for  unimodal,  bounded,  sym¬ 
metric  densities  having  zero  means.  VVe  now  list  some  properties  which  are  relevant  to  this 
discussion: 

1)  The  log-likelihood  ratio  is  odd  symmetric  about  « /2  [that  is, 
((z)  =  log\f  (x-«  )//  (z)l  =  -l(«-z)j. 

2)  Elf  l{X)  —  -Ei;  l(X)  where  the  subscripts  indicate  expectation  under  the  given 
hypotheses. 

3)  varff  1{X)  =  varif  I  (A' ) 

4)  Under  certain  mild  restrictions,  the  variance  of  the  test  statistic  1{X)  is  finite. 

Proofs: 

1)  /  («  -T )  =  log  /  )  =  log  =  -f  (* )  by  the  symmetry  of  /  . 

2)  E„  i(X)  =  f/{x)/(x)dx=-fi(s-x)/(x)dx  =-fi(r)f(r-0)dr  i(x) 

3)  E„  1%X)  =  Jl^(X)f  {X)dX  =  Jl^[s-X)f(X)dX  =  fl^(Y)f{Y-»)dY=EK  l^X) 

4)  proven  in  the  Appendix. 

Generalized  signal-to-noise  ratio  (SNR).  The  motivation  for  defining  a  SNR  for 
detectors  is  the  same  as  that  for  doing  so  with  any  device;  we  want  to  know  how  strongly 
the  output  is  affected  by  the  signal  and  by  the  noise.  For  the  linear  detector  the  two 
effects  are  completely  separable  and  the  resulting  SNR  is  only  a  function  of  the  input  SNR 
and  the  gain  of  the  processor.  For  example,  if  the  noise  is  Gaussian  with  zero  mean  and 
variance  <t*,  then  /■  N. 

(^)  . 


k  ..  1 


-v  •“•y’v'vlviv  .nw"--' 


•  -A 

•’v’ 


where  <I>  is  the  standard  normal  cumulative  distribution  function.  Since  the  <t>  function 
is  monotonic  in  its  argument  it  is  clear  that  for  fixed  a,  0  is  a  monotonic  function  of 
\'ns  !a  .  However,  for  nonlinear  detectors  we  no  longer  have  this  separation  property 
and  the  intuition  which  lends  itself  to  the  linear  case  is  no  longer  evident.  What  is  clear  is 
that  if  we  are  to  use  the  SNR  as  a  measurement  of  detector  performance  then  we  want  a 
higher  SNR  detector  to  have  a  higher  probability  of  detection  at  a  given  false  alarm  rate 
than  a  detector  with  a  lower  SNR. 

The  following  definition  has  been  suggested[l]  for  a  generalized  SNR  for  detectors: 

where  g  is  the  processing  non-linearity.  The  numerator  represents  the  shift  in  the  mean 
between  the  two  hypotheses  while  the  denominator  represents  the  noise  power  at  the  out¬ 
put  under  the  hypothesis  H  (we  have  shown  earlier  that  the  output  variance  is  equal 
under  both  hypotheses  for  the  log-likelihood  detector).  The  definition  has  an  intuitive  feel 
to  it  and  while  it  is  not  always  monotonic  in  ^  for  a  fixed  a  we  will  show  that,  for  a  large 
enough  sample  size,  it  will  be.  Hence,  asymptotically,  this  SNR  is  equivalent  to  the 
Neyman-Pearson  criterion  for  quality  of  detection. 

Some  properties  of  the  SNR[3]. 

SNR  (  k  tampUa  )  =  \fk  X  SNR  (  1  sample  ). 

SNR  is  unaffected  by  gain  or  shift.  That  is, 


1) 

2) 


SNR  [aj(X)-l-  6)=  SNR  (i;(X] 


a  >0. 


3)  For  the  log- likelihood  ratio: 

SNR  \l(X)]  = 


_2£Kjj£) 
var  /(X  )]'/=* 


n.  SNR  AND  THE  CENTRAL  LIMIT  THEOREM  FOR  LOG-LIKELIHOOD  RATIOS 


The  detector  of  Fig.  (1)  is  in  the  form  of  a  sum  and  hence  if  log{f  (JQ-* )//  (A^  )  can 
be  shown  to  have  finite  variance  we  can  use  the  Central  Limit  Theorem  to  determine  the 
asymptotic  properties  of  /(X)  .  In  this  section  we  investigate  the  relationship  between 
SI^  and  detector  performance  for  sample  sizes  which  are  large  enough  so  that  the  Cen¬ 
tral  Limit  Theorem  can  be  invoked. 

In  [3]  it  is  shown  that  if  the  two  hypotheses  are  equally  likely  and  a  =  1-0  then, 
asymptotically,  the  probability  of  error  is  monotonically  decreasing  in  SNR.  We  generalize 
this  statement  with  the  following  theorem. 

Theorem  1:  Given  a  false  alarm  rate,  a,  a  symmetric,  bounded  unimodal  density,  f  , 
and  a  detector  structure  as  shown  in  Fig.  1.  Suppose  we  are  comparing  any  detector  g  for 
which  varxg(X)  >  var^giX)  with  the  linear  detector,  g{X)  —  X,  and  suppose  the  sample 
size  is  large  enough  that  the  Central  Limit  Theorem  can  be  invoked.  Then  a  sufficient 


condition  for  higher  probability  of  detection  of  a  positive  constant  in  noise  with  density  f 
ts  higher  signal-to-noise  ratio.  If  vari^g{X)  =  vacfi  g  {X )  then  the  condition  is  also  neces¬ 
sary.  Furthermore,  if  we  are  comparing  two  log-likelihood  detectors  and  the  Central  Limit 
Theorem  can  be  invoked  then  higher  SNR  is  a  necessary  and  sufficient  condition  for  higher 
probability  of  detection. 

Proof  (see  Appendix  A) 

In  [3]  it  is  shown  that  for  and  a  =  1-^  the  magnitude  of  the  difference  in 

the  probability  of  error  of  the  actual  detector  and  the  CLT  approximation  to  it  is  0(n'*/^) 
and  that  the  relative  error  can  actually  grow  exponentially  in  n.  Here  F;/andP^-  are  the 
prior  probabilities  under  the  two  hypotheses.  This  is  a  somewhat  discouraging  result  and 
elucidates  the  need  for  care  when  invoking  limit  theorems  in  finite  sample  problems.  In 
Part  Ill  we  will  look  at  some  graphical  comparisons  between  the  operating  characteristics 
for  some  detectors  and  their  CLT  approximations. 

The  Central  Limit  Theorem  and  the  log-likelihood  ratio.  We  now  must 
determine  whether  the  log-likelihood  ratio  obeys  the  CLT  so  that  we  can  apply  the  above 
theorem.  We  will  show  that  for  symmetric,  unimodal  and  bounded  densities  having  zero 

mean,  if  Hm  j  '  (z)  <  oo  ,  then  the  log-likelihood  ratio  has  finite  variance  and  the  CLT 

»  -*00 

holds.  More  formally  we  have  the  following  theorem; 

Theorem  2: 

Suppose  X  has  the  density  f  (z ),  where  f  is  symmetric,  unimodal,  bounded  and  has  zero 
mean.  Then  the  variance  of  the  log-likelihood  ratio  g  (x)  =  log  {f  {x~s)/ f  (x)]  is  finite  if 
lim  ff  '  (z)  <  oo  .  Furthermore,  if  X  has  density  h  where  h  is  also  symmetric,  unimodal. 

X  -♦cv; 

bounded  and  has  zero  mean  then  g{x)  has  finite  variance  if  there  exists  Y  such  that 
^  ^1*  ^i.^2  >  0  y  >Y. 

Proof  (see  Appendix  B) 

in.  NUMERICAL  RESULTS 

Rate  of  convergence  to  CLT.  In  order  to  study  the  rate  of  convergence  to  the 
results  predicted  by  the  CLT  we  have  computed  the  ROC  curves  for  the  actual  detector 
and  for  a  CLT  approximation  of  its  output  for  a  variety  of  cases.  In  all  cases  the  detector 
was  optimal  for  the  noise  input  density,  that  is,  g(x)  =  log  [f  {x-»)/ f  (i)]  where  /  is  the 
density  of  the  noise.  The  densities  used  were  three  members  of  the  Johnson  System  fam¬ 
ily  [5]  and  Laplace,  or  double  exponential  noise.  Some  of  our  results  are  shown  in  Figs.  2-8. 

Figs.  2  and  3  show  the  actual  detector  ROC  and  the  CLT  approximation  to  it  for 
Johnson  Noise  with  parameter  6=3  for  two  and  four  samples  respectively.  There  is  a 
marked  improvement  in  the  quality  of  the  approximation  with  the  increase  in  the  number 
of  samples.  Figs.  4  and  5  show  the  same  quantities  except  6  was  changed  to  10.  Again 
there  is  improvement  although  it  is  not  as  dramatic.  It  should  be  noted  that  in  the  limit, 
as  6-^00,  the  distribution  becomes  normal.  Hence  we  would  expect  the  CLT  approximation 
to  improve  as  6  is  increased.  Fig.  6  shows  the  ROC  and  its  approximation  for  5=50.  It  is 
obvious  that  the  approximation  in  this  case  is  very  good. 

In  [8]  a  functional  expression  is  given  for  the  distribution  of  the  output  of  the  log- 
likelihood  detector  for  Laplace  Noise.  From  this  expression  we  have  calculated  the  ROC 
curves  for  this  detector  and  the  CLT  approximation  to  it.  The  results  are  shown  in  Figs. 
7  and  8  for  five  and  ten  samples,  respectively.  Here  again  w'e  see  improvement  with  an 
increase  in  the  number  of  samples  although  it  is  not  as  large  as  in  Figs.  1  and  2.  This 
might  be  due  to  the  fact  that  the  log-likelihood  ratio  for  Laplace  noise  places  a  finite  mass 
at  \'2»  i<f  which  causes  the  output  distribution  to  be  discontinuous. 
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IV.  CONCLUSIONS  AND  TOPICS  FOR  FURTHER  STUDY 


Conclusions.  It  is  clear  that  in  the  limit  SNR  and  the  Neyman-Pearson  criterion 
for  detector  performance  are  equivalent.  The  question  of  whether  SNR  is  an  appropriate 
measure  for  finite  sample  size  detection  is  much  murkier.  As  was  mentioned,  an  upper 
bound  on  the  magnitude  of  the  error  in  predicting  the  probability  of  error  is  derived  in  [3]; 
however,  as  was  demonstrated  by  the  numerical  examples,  good  results  can  be  obtained  in 
certain  cases  when  the  CLT  is  used,  even  in  the  case  of  very  small  sample  size.  From  this 
we  conclude  that  care  should  be  taken  when  using  SNR  as  a  measure  of  detector  perfor¬ 
mance  because  the  accuracy  of  the  resulting  predictions  is  application  dependent. 

Topics  for  further  study.  In  [6],  Czarnecki  designed  a  simple  piece-wise  linear 
processor  for  small-signal  detection  by  maximizing  the  efficacy  over  a  class  of  such  proces¬ 
sors.  Owing  to  the  similar  functional  forms  of  efficacy  and  SNR  an  analogous  design  might 
be  produced  by  maximizing  the  SNR  over  some  class  of  simple  detectors. 

Another  course  of  action  which  might  yield  interesting  results  is  attempting  to  find 
an  error  bound  for  CLT  approximation  for  a  given  class  of  densities  processed  by  a  given 
class  of  detectors.  Both  the  Johnson  System|5)  and  the  Generalized  Gaussian[9]  families 
would  lend  themselves  to  this  endeavor. 

APPENDIX  A 


Proof  of  Theorem  1;  Given  two  detectors,  jj  and  g2-  Let 

mu  =  JgAX)fj(X)dX  .  =1,2  J  ^H,K  , 

au^  =  f[g,{X)-mu\"fj{X)dX  , 


where  Jh{^)  =  /  (* )  ^^nd  fK(^)  =  /(*-*)•  Since  we  assume  the  CLT  holds,  the  density, 
under  the  hypothesis  J ,  of  the  test  statistic. 


a(x)= 


is  normal  with  mean  nniu  and  variance  n<r,7^  .  We  then  have: 

timm  -  t, 


a,  =  — — -  /  «  ^  dz  —  i — 

V2.r/  '  v/ba.-; 


and 


1  .  ,  At  ******^ 

B:  =  I  e  dz  =  <1> — — - 

\/n  ffiK 


where  once  again  <l>  is  the  standard  normal  cumulative  function. 

Equating  the  expressions  for  oi  and  qj  and  solving  for  yields: 


<  2  =  ( L  ~  Iff  ) - 2ff 

•^iff 


Substituting  for  tj  in  the  expression  for  ^  yields: 


02  =  * 


,  ^2H 


"ff»2ff  Iff) - '»*”2ff 


•Jn  a^K 


(A.1] 


(A.2; 


We  now  turn  our  attention  to  the  case  where  g^  is  the  linear  detector;  that  is, 
((i(A')  =  A'.  We  then  have: 


<^2H 


' \K 


*  ’  Iff  =  0,  OiH  —  (Z\K  —  <^1,  *2  —  -  ■*"  ’•"•2ff 


<1.  — - !-  ,;92  =  <1> 

I  vn  ffi  j 


nmo^-  -  1 1 -  nvir 

<^i 


Then  02>0i  if  and  only  if 


.  ^2H 

rtni2A--<i - nm^H 

ffj  m  -  1 1 

^2K  1 

After  rearranging  we  have: 

Sjv;p{^  _  ”‘2A'  -  f"2H  ^  fn2K  -  ">2H  ^  J_I_  "  ‘'2H  ) 

02H  ~  O2K 

where  the  second  term  on  the  right  side  is  non-negative.  Hence  the  condition 

SNlRo  >  — 

<^1 

is  sufficient  for  02>  0^-  If  (T2){  =  02H  Eq.  (A. 3)  reduces  to 


^2K  -  'f>^2H 


SNRj  >  —  , 


and  the  condition  in  Eq.  (.A. 4)  is  also  necessary. 

For  the  log  likelihood  ratio  we  have  <r,//  =<7, a-.  From  Eqs.  (A.l)  and  (A. 2)  we  have 
0.,  >  0i  if  and  only  if 


”"‘2ir  -  (^1  -  1W)“ - , 

_ ^  nmijc  -  I| 

<^2/f  ‘^IK 

.\fter  some  rearrangement  we  have 

2A'  “  2«  ^  ^2H  *  1  ,  1  <'2H  > 

-  >  mxH - ( - )• 

(’2K  <^IH<^2K  «  O'lA'  <^\H^2K 

Under  the  equality  constraint  on  the  variances  this  reduces  to 

CN'D  "»2*  -  "»2//  mxK  -rtliH 

SNRo  =  -  >  - -  SNRi  . 

(72  <7l 

APPENDIX  B 

Proof  of  Theorem  2:  We  are  interested  in  showing  that  the  following  two  integrals 

converge 


E  ^g\X)!(X)dX 

-00 
+  00 

E  9  =  i9{X)l{X)dX 


If  Ijfr)!  =  \l09\f  (x-t)l }  {x)]\  is  bounded  (<  M)  then  we  have 

E  ?='  =  <  M^jf  =  W* 

and 

E  g  =  jgf  <  Mjf  =  M  . 

Hence  for  bounded  g  we  have  bounded  variance. 

We  now  look  at  the  case  where  g  is  not  bounded.  In  order  to  show  that  the 
integrals  in  Eqs.  (B.l)  and  (B.2)  converge  we  want  to  bound  g  and  /  with  known  func¬ 
tions  for  which  convergence  can  be  demonstrated.  Since  the  problem  is  entirely  symmetric 


-V .v_v  V  V 
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we  will  concern  ourselves  only  with  integration  from  +—  to  infinity.  For  a  Gaussian  den¬ 
sity  with  variance 

2 

and  for  a  double  exponential  density  /  (r )  =  *  I  with  variance  r 

T 

-\/2—  if  X  <0 

a 

\/2 

guv  =  - (2ar-s)  if  0<i<« 

^  <T 

—  if  x>» 

a  ~ 

Hence  for  any  unbounded  log-likelihood  ratio  with  lim  g  '  (z)  <  oo  and  for  all  z  >  P 


we  can  say 


gupj^}  <?{*)<  )  . 


where  cti  ,<j2  and  P  are  positive  constants.  That  is,  g  is  bounded  above  by  a  straight  line 
with  positive  slope  and  below  by  a  positive  constant.  We  then  have 

9(x)<^-:^  (B.4) 

<^1  2<t,2 


s{-r)  > 


for  all  z  >  P  .  From  Eq.  (B.5)  we  have 


>  «  * 


We  then  have 


/U)  <  «  /u-*) 


-V2» 

/  (O  <  *  "*  /  (0) 

/(2s}<  e  "’/(«)<«  /  (0) 

-y/2i  -t  y/it 

f{k»)<  «  /  |(*-1)»1  <  e  /(O) 

where  due  to  the  previously  mentioned  constraints  on  /  we  know  it  has  a  finite  maximum 
at  z  =0  .  We  now  define  the  step  function  /  ,  as  follows 

-(*-llv^» 

/,(i)=e  'a  /(O)  {k-l)B<x<ks  t  =  1,2,3,.,. 

/  .(0)  =  /  (0)  . 

This  function  dominates  /  .  We  now  go  one  step  further  and  define  /  j  as 


/2(z)  =  r  =  /(O) 


z  >  0  . 


This  expression  can  be  written  as 


/  ■:(■*• )  =  f  i«  X  >  0  .  (B.7) 

where  <•  i  and  cn  are  positive  constants.  It  is  this  function,  which  dominates  both  /  ,  and 
/  ,  that  we  use  to  prove  the  integrals  in  Eqs.  (B.l)  and  (B.2)  converge. 

Since  both  /  and  g  are  continuous  they  attain  a  maximum  in  anv  closed  and 

p  P 

bounded  set'";.  Hence  both  f g^(z)f  (i)dT  and  fg(x)f  {z)dx  are  finite.  From  Eq.  (B  3)  it 

-1 

2  2 

is  clear  that  if  we  can  show  ear  2-nd  E  2re  finite  then  the  proof  is  complete. 

oo  oo 

This  is  implied  by  showing  that  Jzf  2(z}dz  and  Jz"/  2{z)dz  converge.  But 

p  p 


J c  ^xe  dz  =  — -e  ‘^{P  +  -^)  <  oo 
p  <"2  r  2 


-2^/02  .  2P  2 


Jc,T‘e  =  — -e  (P^  +  — h  -^)  <  oo 


C2 


f2  f  2 


and  the  theorem  is  proved. 
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